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Our  purpose  was  "to  study  a  variety  of  independent  approaches  to  the 
analysis  of  human  judgment  and  decision  making  with  the  aim  of  integrating  these 
approaches  into  a  coherent  body  of  knowledge"  (p.  2,  CU  Proposal 
No.  0277.8.091B,  June,  1977).  There  seemed  little  doubt  about  the  need  for  such 
an  attempt  in  view  of  the  readily-documented  fact  that  the  "current  literature 
bearing  on  judgment  and  decision  making  contains  thousands  of  entries  produced 
by  professionals  in  such  different  disciplines  as  psychology,  management 
science,  operations  research,  planning  and  policy  sciences,  sociology, 
statistics,  systems  analysis  and  others"  (p.  2,  Proposal).  Moreover,  the 
proposal  asserted,  "the  authors  of  these  articles  seldom  acknowledge  the 
presence  of  studies  outside  their  own  discipline,"  and  therefore,  these  isolated 
research  efforts  have  resulted  in  uneven  development,  and  a  discipline  that 
"continues  on  an  unpredictable  course  .  .  .  characterized  by  redundancy  of 
effort  on  one  hand  and  numerous  unexplored  problems  on  the  other”.  We  found 
nothing  in  the  four-year  pursuit  of  our  topic  that  is  inconsistent  with  the 
views  expressed  above.  Nor  did  we  find  any  effort  other  than  our  own  to  remedy 
what  virtually  everyone  will  acknowledge  to  be  an  unsatisfactory  state  of 
affairs. 
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Because  the  main  substance  of  our  work  is  fully  described  in  two  reports, 
both  of  which  have  been  published  in  book  form  and  already  distributed  to  those 
receiving  this  Final  Report,  in  what  follows  we  briefly  indicate  the  methods 
employed,  and  the  principal  results  of  our  work,  rather  than  presenting  details. 

Methods 

Our  first  step  was  to  cut  our  ambitions  to  fit  our  professional  and 
budgetary  resources.  Rather  than  including  in  our  study  work  from  all  of  the 
disciplines  mentioned  above,  we  focused  our  efforts  on  six  major  approaches, 
anchored  at  one  end  by  the  largely  analytical,  mathematically-oriented  approach 
taken  by  Keeney  and  Raiffa  and  at  the  other  by  the  largely  psychological 
( nonmathematical ) ,  empirically-oriented  approach  taken  by  Jones,  Kelley  and 
others,  known  as  attribution  theory.  We  also  included  in  our  study  the 
approaches  taken  by  Ward  Edwards,  Daniel  Kahneman  and  Amos  Tversky,  Kenneth 
Hammond,  and  Norman  Anderson. 

We  chose  to  focus  our  work  on  approaches  to  theory  and  research  rather  than 
on  the  description  of  the  judgment  and  decision  process ,  for  the  reason  stated 
at  the  outset:  There  is  no  general  agreement  on  what  form  such  a  description 
should  take.  For  us  to  have  selected  one  among  the  many  possible  descriptions 
of  the  judgment  and  decision  process  that  have  been  put  forward  would  have  put 
us  in  a  position  of  advocacy,  and  would,  therefore,  have  compromised  our  efforts 
to  achieve  a  descriptive  framework  that  was  theoretically-neutral .  The  first 
year  of  the  project  was,  therefore,  devoted  to  constructing  a 
theoretically-neutral  descriptive  framework  and  applying  it  to  the  six  major 
approaches  indicated  above. 
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Results 


Year  Is  (1977-78) 

A  comparative  study  of  the  theory,  methods  employed  and  procedures  used  by 
six  major  approaches  was  undertaken .  The  results  of  the  study  were  described  in 
a  report  mailed  to  twenty-seven  established  researchers  in  the  field  who  were 
then  invited  to  participate  in  a  conference,  the  purpose  of  which  was  to 
criticize  and  improve  the  report.  At  least  two  persons  who  were  representative 
of  each  of  the  six  major  approaches  were  present.  A  critique,  prepared  prior  to 
the  conference,  was  presented  by  one  person  representing  each  approach.  Others 
presented  critiques  as  well.  The  principal  outcome  of  the  conference  was  the 
recommendation  that,  because  of  its  potential  usefulness,  the  report  be 
published  in  book  form.  This  step  was,  therefore,  undertaken  in  Year  II. 

Year  II:  (1978-79) 

Three  main  steps  toward  integration  were  taken  in  this  period:  (a)  the 
report  written  during  the  first  year  was  reviewed,  revised  and  sent  to  the 
publisher;  (b)  representatives  of  the  six  approaches  studied  in  Year  I  were 
presented  with  the  same  two  practical  problems  and  comparisons  were  made  of  the 
methods  employed  by  them  to  cope  with  the  problems;  (c)  a  detailed  analysis  of 
differences  between  theory,  method  and  procedures  of  two  of  the  six  approaches 
was  made. 

As  in  the  first  year,  reports  regarding  the  latter  two  projects  were  mailed 
to  established  researchers  who  were  then  invited  to  a  conference  to  criticize 
and  improve  the  work.  The  general  reaction  to  these  reports  ran  from 
"indifferent"  to  "negative".  The  second  conference  failed  in  the  sense  that  the 
reports  that  were  presented  were  judged  as  not  having  served  the  purposes  of 
integration  well.  One  constructive  conclusion  was  reached,  however,  when 
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several  members  of  the  conference  concluded  that  lack  of  a  common  language  was  a 
barrier  to  integration.  They  contended  that  a  glossary  was  needed  and  that  it 
should  be  constructed  by  this  research  group  since  a  glossary  was  needed  to 
further  the  aims  of  integration.  The  investigators  decided  that  this  conclusion 
was  correct:  The  major  concepts  in  the  field  should  be  listed,  authoritative 
definitions  provided,  their  origins  should  be  found,  noted  and,  comments 
provided  when  appropriate.  As  a  result,  this  project  was  undertaken  in  the  fall 
of  1979. 

Year  Ill;  (1979-80) 

During  this  period  the  report  described  above  was  published  in  book  form 
under  the  title  "Human  Judgment  and  Decision  Making:  Theories,  Methods  and 
Procedures".  (The  report  on  which  the  book  was  based  had  already  been  sent  to 
each  person  whose  name  was  on  the  OMR  distribution  list.) 

Work  continued  on  the  glossary  throughout  the  year.  In  addition,  a 
subcontract  was  let  to  Drexel  University  to  carry  out  a  co-citation  analysis  of 
publications  in  the  field  in  order  to  test  the  validity  of  the  manner  in  which 
the  work  of  many  authors  was  organized  into  six  approaches  in  the  first  book, 
and  to  present  an  empirically-based  pictorial  structure  of  research  groups  in 
this  field. 

The  following  new  projects  were  also  undertaken: 

1 .  Hammond  developed  a  general  theoretical  framework  for  the  purpose  of 
unifying  research  and  theory  in  the  field  (see  CRJP  Report  Mo.  226). 

2 .  McClelland  undertook  an  axiomatic  approach  to  the  concept  of  weight 
(see  CRJP  Report  Mo.  227). 
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3.  Shanteau  compared  the  use  of  the  concept  of  weight  in  two  approaches 
(see  CRJP  Report  Ho.  228). 

Year  IV:  (1980-81) 

During  this  period,  the  glossary  was  completed  in  book  form  and  punlished 
under  the  title  "Concepts  in  Judgment  and  Decision  Research:  Definitions, 
Sources,  Interrelationships,  and  Comments".  The  co-citation  study  by  Drexel 
University  verified  the  validity  of  the  organization  of  approaches  to  research 
described  in  the  first  book  and  provided  a  useful  pictorial  structure  of  the 
field.  The  results  of  the  co-citation  study  were  therefore  included  in  the 
glossary.  The  glossary  was  issued  by  the  publisher  on  October  15,  1981  and  a 
complimentary  copy  was  sent  to  all  persons  whose  names  appear  on  the  Center  for 
Research  on  Judgment  and  Policy's  OHR  distribution  list. 

In  addition,  theoretical  efforts  toward  unification  continued  during  the 
year.  Hammond  described  the  similarities  and  differences  in  problem-solving  and 
judgment  and  decision  making  research  (see  CRJP  Report  No.  231).  And  the 
empirical  analysis  of  100  research  articles  be gam  to  produce  useful  knowledge. 
For  example,  it  is  clear  that  researchers  show  a  surprising  lack  of  research 
concern  for  variables  often  mentioned  as  important  in  cognitive  research.  In  a 
report  not  yet  complete,  Hammond  and  Quinn  found  surprising  gaps  in  the 
reporting  of  information  regarding  the  properties  of  the  tasks  used  in  judgment 
and  decision  research.  After  examining  88  articles  published  within  the  six 
major  approaches  described  in  Hammond,  McClelland,  and  Mumpower  (1980),  as  well 
as  12  other  papers  not  included  within  these  six  approaches,  they  found  the 
following  omissions  regarding  task  properties:  (a)  14%  of  the  articles  do  not 
report  the  number  of  judgment  scale  levels;  (b)  when  cues  are  simultaneously 
presented,  24%  do  not  report  whether  or  not  relevant  cues  are  identified  for  the 
subject?  (c)  95%  do  not  report  the  amount  of  time  given  to  the  subject  to 
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respond;  and  (d)  when  an  outcome  criterion  is  present,  34%  do  not  report  the 
type  of  relationship  between  cue  and  criterion,  or  the  ecological  validities 
(weights)  of  the  cues,  or  the  organizing  principle  in  the  task,  among  other 
omissions.  With  regard  to  results,  (a)  20%  fail  to  report  the  subjects' 
consistency  over  time  or  trials;  (b)  90%  do  not  report  the  distribution  of 
errors;  (c)  98%  do  not  report  and/or  investigate  subjects'  reports  regarding 
judgment  processes;  (d)  80%  do  not  report  response  time;  (e)  84%  do  not 
investigate  or  report  the  subjects'  level  of  confidence;  (f)  60%  do  not  report 
which  judgment  parameters  change  over  trials;  (g)  25%  do  not  report  the 
weights,  cue-criterion  relationships,  or  organizing  principles  used  by  subjects; 
and  (h)  90%  do  not  inquire  into  memory  processes,  among  other  omissions. 

The  complete  results  of  this  analysis  will  be  distributed  in  report  form 
during  the  year  1981-82. 

Shanteau  used  computer  simulation  techniques  to  study  the  conditions  under 
which  the  synergistic  use  of  information  would  fail  to  be  detected  by  the 
investigator  (see  CRJP  Report  No.  235). 

Conclusion 

Zt  will,  of  course,  be  impossible  to  be  certain  about  the  results  of  these 
efforts  toward  integration  in  the  immediate,  or  even  the  distant,  future.  Our 
books  and  reports  will  be  more  likely  to  affect  the  graduate  students  of  the 
1980 's,  rather  than  present  investigators.  And,  of  course,  there  will  be  many 
new  research  developments  in  the  1980 's  aside  from  our  books  that  will  gain  the 
interest  of  students  and  thus  shape  their  development;  the  causes  of  change 
will  be  difficult  to  determine.  It  seems  unlikely,  therefore,  that  it  will  be 
possible  to  discover  a  direct,  empirical  link  between  the  work  carried  out  here 
and  our  aim  of  "integrating  these  approaches  into  a  coherent  body  of  knowledge". 
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Therefore,  unless  and  until  an  evaluation  is  undertaken  by  means  of  a  survey  of 
graduate  students,  say,  the  books  and  reports  produced  by  this  project  will  have 
to  be  judged  on  their  own  merits  as  contributions  to  the  organization  and 
development  of  the  field  of  judgment  and  decision  making. 
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